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PREFACE 



In the past few years countless authors have suggested, in one way or 
another, that cost-effectiveness methods be used as a basis for decision- 
making in education. As one sorts and analyzes these proposals and prescrip- 
tions two important inadequacies become evident: 

1) Firm theoretical bases for these studies are lacking. Theory, in the 
domain of decision-making, should provide not only a basis for 
description and explanation but explicit statements of assumptions 
underlying the proposed rationale and methodology. 

2) Little help is offered to the individual who wishes to select from 
the various economic based approaches an appropriate method to 
apply to a practical problem. 

This monograph is written to meet needs arising from these inadequacies 
in the literature. It is anticipated that this monograph will be utilized 
by a limited technical audience who is interested in: general theory of 

school system planning, and the theoretical substantiation for benefit- 
cost analysis and cost-effectiveness analysis. Those who want to design 
realistic methods for school district planning may also find assistance here. 
Above all, this is directed to those who believe that meaningful change can 
be effected in schools by systematic and rational planning. Hopefully, better 
planning methods will be followed by higher levels of educational productivity 
for the educational dollar. 

The bulk of this document is devoted to the theoretical development of 
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nine decision cases* A three page summary of the nine cases offers 
specific recommendations about methods which can be used for gener— 
ating and processing the kind of "data types" required for educational 
decision-making . 

An appendix provides a discussion of some combinatorial problems. 
A bibliography, which was used in developing many of the key concepts, 
is also included# 

. % \ 

Numerous people have contributed to the synthesis of the ideas in 
this paper. It is not feasible to mention them all, but a few must be 
singled out for their impact on the author’s thinking. I shall list 
these names and hope that each understands how much his help is appre- 
ciated: Professors Francis Brown, Morris Hamburg, Donald Morrison, 

and George Parks of the University of Pennsylvania; Professor Harold 
Goldman of Bucks County Community College; Paul Hood, Director of the 
Communication Program of the Far West Laboratory for Educational 
Research and Development; Roger Sisson and Robert Pritchard, consul- 
tants; and James Becker, Louis Maguire, Harris Miller, Joseph Mir sky , 
Frederick Tanger, and Jo Ann Weinberger, all of Research for Better 
Schools. 



S. T. 
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DECISIONS AND CHANGE: THE ROLES 



The theory developed in this paper is quice general and may be applied 
to systematic study in many areas. The local school district has been 
selected as a situation to be used for specific comments to facilitate better 
communication . 

There are several people who perform ro].es in the decision-making pro- 
cesses which are described, it is important for the reader to become acquaint 
ed with these roles so that their inter-relationships can be appreciated. 
Decision-maker 

First, there is the decision-maker . He is the person who is responsible 
for decisions. At times he may delegate his authority to another person, but 
the responsibility is ultimately and rightly his. The role is fitting for 
the superintendent of schools. 

Educational Engineer 

Second, we should understand the role of the educational engineer . He 
is responsible for designing programs which meet the "overall objectives" of 
the school district. Often the engineer may design several alternative pro- 
grams each of which is designed to meet same set of targets or overall ob- 

♦ 

jectives. The superintendent may want to select the best alternative from 
the group supplied by the engineer. In formulating the alternatives, the 
engineer must assure that they are practical and can be implemented within 
the framework of available resources. A change may be technically sound, but 
when implementation personnel are not equipped with the understanding, train- 
ing, and attitudes required to effect the change, there is little chance that 
the change will be effective. 
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Engineer versus Professor-scientist 

The engineer's role is often taken by a committee of teachers, 
a high school principal, and, on occasion, the superintendent himself. 
Sometimes a school district hires a professor-scientist from a uni- 
versity. The professor-scientist is often interested in an experi- 
mental situation in order to obtain information to test his theory 
for subsequent journal publication. The school district's engineering 
needs, then, may not be consistent with the role played by the professor 
scientist, although this is sometimes obscured. 

Implementor 

V 

Another role is that of the implementor . He is responsible for 
implementing changes suggested by the decision-maker. He directs 
his staff, the implementation team . This implementation role is, in 
the opinion of the author, taken too much for granted by everyone, 
including those involved in implementation. The implementor is some- 
times a school principal, a curriculum coordinator, a research office 
person, or the engineer himself. 

Evaluator 

Th^ last role to be described in this introduction of characters 
is that of the evaluator . To understand the evaluation role one must 
first appreciate that there are many reasons for undertaking evalua- 
tion. Some of these are: 1) the funding agency insists that evalua- 

tion be conducted, 2) the decision-maker feels that evaluation is 
important, 3) the research office feels that evaluation is important, 

4) the implementor feels that evaluation may be helpful in implemen- 
ting the program in the next period, etc. 
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Evaluation is conducted for the purpose of decision-making. Evaluation 
may focus on providing information to help the engineer design better programs; 
in the future, or to aid the decision-makei in his assignment of priorities 
or to assist the implementor in improving his implementation techniques. 

In addition it may indicate the need for better communications among decision- 
maker, engineer and implementor. If, however, the evaluation role does not 
enter into the decision-making process in specific, recognizable ways , then 
it is not consistent with the posture suggested in this paper. 

TIME, ACTIVITY, AND LEVEL OF RESOURCES 

The decision framework used in the development of theoretical arguments 
in this paper is an evolutionary one. Primitive cases introduce much of 
the terminology. As the cases become more realistic the terminology takes 

on added complexity. In order that the reader can appreciate the decision 
framework, some supporting concepts are now introduced. 

Time Reference 

Two time references are considered in this paper. An a priori , or prior, 
reference is required when the decision-maker wants to: 

1) Decide among alternatives without considering an existing 

school system: this would be appropriate when considering 

designs for a completely new school system. 

2) Decide among alternatives for an existing school system: 
this would be appropriate when considering changes for a 
school system. 



I ‘ "! ijm ^ if**** * ~ I *. *v&r*t**.- i . - ~ ' --, «- — *, *< -* - ** » >•* ^«v» **r --. *- •v*"-, < >»v>yw ■ .^m** m* ^* r? - >*«, | ! ■ <icy«q | n^y y / np ^iy n ' mq 
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An a. posteriori , or evaluative, reference is used when the 
decision-maker wants to evaluate how well the school system did in 
relation to its stated objectives. 

> ■*’ 

Activity 

An activity-design is a plan for an activity. It is developed 
by the engineering function and lends itself to a priori decisions. 

Once an activity-design has been selected by the decision-maker and 
implemented, it generally will be evaluated. Evaluation of an ac- 
tivity-implementation is carried out to supply information useful 
for a new cycle of decisions. 

Some activity-designs are very complex. When complexity mandates 
more detailed analysis, the notion of task is introduced. A set of 
tasks comprises a program-package in more complex instances. Some of 
these tasks arc independent of performances of preceding tasks 
("in parallel"), while other tasks are dependent on performances 
of preceding tasks ("in series") . 

Finally, the term activity is used in later sections of the 
presentation. Activity connotes the same meaning as program-package 
but is more meaningful given the context. 

Level of Resources 

Cost is viewed by the decision-maker in one of three ways. The 
level of resources is either unlimited, limited or known. Unlimited 
and limited refer to a. priori situations and known refers to a. posteriori 
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situations. 



THE NINE CASE FRAMEWORK: AN OVERVIEW 



The cases described in this section, as indicated earlier, pre- 
sent an evolutionary framework. The apparent nature of this evolution is 

toward complexity; yet while this is true, there is a more fundamental 
flow. The underlying feature of the nine case structure is realism. Case 
1 represents the simplest decision situation possible. It is also remote 
from the reality of school systems. Cases 8 and 9 represent complex sit- 
uations. Their solutions represent, as will be seen, prescriptions for 
school district planning and decision-making. 

Each case is developed and analyzed in the format which follows: 

Case structure 

1. Number of objectives (single or multiple). 

2. Number of activities (single or multiple). 

3. Time scale (a priori , i.e. activities are to be selected, or a_ 
posteriori , i.e., activities were selected and implemented and 
are to be evaluated) . 

4. Level of resources (unlimited or limited, in the a priori 
cases, or known in the a posteriori cases). 

Case approach 

1. Discussion of the approach taken to develop the decision- 
variable. The decision-variable enables the decision-maker 
to select the preferred course of action or plan. The 

preferred plan is expected to result in meeting more 
of the decision-maker’s objective (s) than any other 



alternative, given the case assumptions and conditions 



Case summary 

Discussion of the flow of information required to support 
selection of the preferred alternative. 

Much of the mathematics presented is used to provide inductive 
proofs consistent with the total approach which should be seen as a 
synthesis emerging from the simplest case — Case 1. Figure 1 out- 
lines the logic underlying the classification of the nine cases. 



FIGURE 1 



A CASE CLASSIFICATION OF LOGICALLY RELATED DECISION PROBLEMS 



STRUCTURE 



CASE 



1. Single objective with 
set of proposed plans 
(activity-designs) 
for achieving the ob- 
jective; one plan is 
to be selected. 

2. Single objective with 
set of activity- 
designs; one to be 
selected. 

3. Evaluation of Cases 
1 and 2. 



4. Sxngle objective with 
set of tasks; several 
to be selected as a 
program package. 

5. Single objective with 
set of tasks; several 
to be selected as a 
program package. 

6. Evaluation of Cases 
4 and 5. 

7. Multiple objectives 
with multiple activ- 
ities; several to be 
selected. 

8. Multiple objectives 
with multiple activ- 
ities; several to be 
selected. 



9. Evaluation of Case 

8 . 



DECISION RESOURCE 

FRAMEWORK LEVEL 

A priori Unlimited 

A priori Limited 

A posteriori Known 
A priori Un.' iited 

A priori Limited 

A posteriori Known 
A priori Unlimited 

A priori Limited 

A posteriori Known 



COMMENT 



A utopian research and 
development problem. 



The constraint limits 
admissible alternatives; 
but still no incentive 
to economize. 

A performance evaluation 
involving a partition of 
outcome space. 

A more complex version of 
Case 1; still utopian. 



The contraint limits 
admissible alternatives 
as in Case 2; still no 
incentive to economize. 

A more complex version of 
Case 3. 

A much more complex version 
of Cases 1 & 4; the relative 
weight of objectives becomes 
important. 

The general cost-effective- 
ness case; the only case 
meeting the necessary and 
sufficient conditions for 
cost-effectiveness analysis. 

The general program evalua- 
tion case; provides cost- 
effectiveness evaluations 
for present year and inputs 
for next year’s budget. 
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CASE 1 

This situation is described by a single objective. The decision- 
maker must assess a group of alternative activity-designs and select one. 

These alternatives have been prepared by an expert, herein referred to as 
an engineer. The time scale is a priori and resources are unlimited. 

The decision-maker is confronted with a "utopian R & D" situation 
and must select the optimal plan to meet the given objective. Optimization 

implies that some type of constraint is put on the decision-maker (if not, 
he would be involved in maximization) . Some constraining factors are time 

(how much time will the decision-maker allow until he wants to see some 
results?), state of technology (does measurement methodology exist to enable 
the implementor to measure a critical variable?), and attitudes of personnel 
(did the engineer have sufficient commitment to the task of designing improved 
alternatives?) . 

Activity-design JL has a distribution of performance estimates , Ki • 

This distribution is based on the engineer* s perception of the performance 
after implementation and hence includes the engineer’s biases. 

The decision-maker’s aim is to select that alternative which provides 
a preferred balance between estimated average performance and variability 
of performance , i.e., he is to use the mean-variance decision-criterion. The 
standard deviation , & , has been used as a measure of risk in studies deal- 
ing with investment decision models.^ - 



^For example, see: Donald E. Farrar, The Investment Decision Under 

Uncertainty (Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1962), pp. 25-26. 
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Baeie- to the mean-variance criterion is the idea that there exists a re- 
lationship between risk,<r\ and acceptable mean return, Most decision- 

makers are risk avoiders. They require increasingly higher mean returns 
as risk is increased, i.e.jthey discount risk at a higher rate than mean return. 

A certainty-equivalent function is an indifference curve comprised of 
combinations of for which the decision-maker is indifferent. The 

certainty-equivalent is the value of mean return corresponding to zero risk 
for a given indifference curve. If the decision-maker is allowed to choose 
from the fa r 3 points on a given curve, including the certainty-equivalent, 
he should be indifferent to all. 

Indifference curves are generally used to indicate combinations of two 
commodities for which a person, given his preferences, is indifferent. An 
indifference curve is negatively sloped reflecting that it takes an increased 
amount of one commodity to offset the loss of an amount of the other commodity. 
The slope of the curve at any point indicates the marginal rate of sub- 
stitution of one commodity for the other. Two indifference curves cannot 
intersect since this would mean that there could exist two equally preferred 
points each with the same return but with different risk assignments. 

How does this relate to the selection of the preferred activity=design? 

9 

Assume that the engineer can provide, for each proposed activity-design X. , 



expected performance, , and variance, ^ 

7k* 



where 



Jx- * JK 
\ 

S- - ( (Ki- 






, - k: — )•* • 
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It is assumed that performance is defined over the range 0 to 1.0. This 
range indicates the degree to which the activity-implementation is expected 
to meet its objective. Initially, the decision-maker selects a minimum 
level of acceptable performance, Kq . The upper limit is based on 

the engineer’s estimate of a level of performance which ’’cannot" be surpassed 

I 

were activity— design A* to be implemented. Once Kq bas been established, 
the decision-maker considers those higher performance scores and their inherent 
risks which he prefers equally to a riskless Kq • These points define an 
indifference curve which is independent of any data received from the engineer. 
That is, the curve is defined solely on the basis of the decision-maker’s 
preference for various combinations of expected performance and risk. If the 
preferences of the decision-maker were not considered, then he would be unnec- 
essary in Case 1 and the engineer could make the selection using his criteria. 

Figure 2 indicates the general nature of the Case 1 decision-maker’s 
indifference curve, denoted • The sketch describes a relationship of 

expected performance to variability for a decision-maker characterized by risk 
avoidance. Therefore, tCC must increase at a faster rate than in order 
for the decision-maker to remain indifferent. K $ is the certainty-equivalent 
value. 



Performance 
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Th e X region of Figure 2 contains points for activity-designs which 



have expected performance levels below the minimum acceptable level, ff 0 . 



performance, but all of these points are dominated by any point on 
having the same expected performance but a lower variability. 

Figure 3 shows a group of indifference curves. The curve 
increases rapidly in the region S' - Q , S’ , indicating the decision-maker’s 
uneasiness over the high level of variability. 



these indifference curves. Since three constants are to be determined. 



The Y region contains points that are acceptable with respect to expected 



Performance 




X 



AO 



Variability 



Figure 3. Several Indifference Curves 



Suppose a parabola of form 




approximates the form of 



three sets of coordinates are required. For instance. 



[ ti, * 0 * , Si ‘ , 
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* 0 . 3 }^ | satisfy the equation 

S ’ O.IO -O.lSK +l lfk 1 • In general, lQ values can be obtained for 
the quadratic form using the formula 



W. - * ^ # c- ^ 

'4 * ' zc 



o . 



Some checks on the equation would involve seeing that O ^$ t ' — I 
and 4z \ . The slope of the function can be interpreted as the 

J)/. 

rate of increase of ^ with , that is j 

K * - 1 - O- 






zc 

-X 



1 1 ) (J l - fcfa-s)]'* (+c ) 






X 



Using the equation for we find the rate of change in expected 

performance per unit of variability at the point S* 0 >£i' the previous 
equation is 0-6^3 (~0 £ X is not meaningful). This means that the rate of 
change of the curve when J-* O«T0is0.5<ik'*per unit of £♦. At £ • * 0 * (q 





Variability 



The logic that allows the preferred point to be selected is not complex. 
The ideal point is g t -0^. Figure 4 shows an indifference curve 

and three points. Points A and B are equally preferred since they lie on 
r c|c .) . Point C is preferred to both A or B since it has a higher perfor- 
mance value than corresponding points on t(k') having the same level of 
variability as Point C. Point C is also preferred to any point on 
having the same level of performance as C since C has less variability. The 
preferred activity-design is the one with a point lying on or above the upper- 



most indifference curve. 

In conclusion, for Case 1 the decision-maker should state his preferences 
as combinations of expected performance and variability . Then he should 
select that alternative which will provide the optimal combination of expected 
performance and variability according to his feelings. 



Source 

decision-maker 
engineer (s) 
engineer (s) 
engineer (s) 
decision-maker 
decisior -'aker 



Case 1 data types by source* 

Data Type 

single objective 

activity-designs 

performance estimates 

distribution of performance estimates 

minimum level of acceptable performance 

indifference curve of K* 0 



£ 

Implied in the development of Case 1 was a performance criterion since 
a prior condition for obtaining a performance estimate , J(* 4 * ^ is that a criterion 
exist. This criterion is prepared jointly by the decision-maker and engineer. 
They may be wise, however, to consult with users of the system and the 
implementor . 
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CASE 2 

This case differs from Case 1 in one important respect: resources are 
limited. Again there is a single objective with multiple activity-designs 
from which only one is to be selected in an <1 priori framework. The decision- 
maker will select that activity-design which promises the highest certainty- 
equivalent he can afford. Wien he does not elect to behave in this manner 
he has implicitly introduced a second objective. It may be a personal objective 
such as the desire to be efficient under any circumstances, or he may entertain 
some vague notion of future possibilities developing for investment and hence 
want to save funds . But the point remains , however , that in Case 2 he has 
no alternative objectives in competition with the attainment of the single, 
given objective. 

The decision rule is to select the activity-design represented by a 
point on or above the highest indifference curve, provvded tbit it fall? 
within the limitations of his budget. Instead of treating the three variables 
(mean, standard deviation and cost) in a three dimensional drawing, the 
decision graph may be viewed as two dimensional by using the certainty- 
equivalent in place of an <* 5X • Figure 5 shows certainty-equivalent 
values on the ordinate and estimated costs associated with each of the 
admissible activity-designs on the abscissa. Costs are assumed to be point 
estimates with no accompanying measure of variability. The decision-maker 
should, therefore, be cognizant of this, since activity-implementations with 
prior cost estimates less than but near may exceed C M/ }g . When there is 

some budgetary leeway he may not be troubled by the possibility of exceeding 
C h<K . In realistic instances the decision-maker tries to provide for such 
contingencies. 




-15- 



Admissible alternatives, in Figure 5, are represented by points for 
which Q ^C/v, 4 X ) and • In order to select the preferred alterna- 

tive, the decision-maker selects the point having the highest certainty- 
equivalent within the region Q If there were two or more candidate 

points having "equally highest" certainty-equivalents, then the decision- 
maker would select the least-cost point, although accoi Ung to the strict 
structure and assumptions of Case 2 he would be unable to use a partially 
unexpended budget. The net result of adding the budget limitation is that 
points such as B in Figure 5 are eliminated from consideration. In the 
Figure 5 example } the decision-maker would be compelled to select Point C 
over Point B if Point A did not exist, indicating the potential importance 
of a budgetary restriction. 



Performance Oca 
Certainty-Equivalent 




Cost 



ic<) 




Figure 5. Selecting a Preferred Point with a Cost Constraint 
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In summary, for Case 2 the decision-maker should follow the guidelines 
established in Case 1 with regard to stating his preferences. Then he should 
select that admissible alternative which will provide the optimal combination 
of expected performance and variability. The cost constraint determines 
which activity-designs, of those promising /T< ^ » are admissible. 



Case 2 data types by source* 



Source Data Type 

presumably imposed on the decision- cost constraint^ 

maker 

engineer and implementor in concert cost estimates C^’ 



*Only new data types are included. Those which were defined in Case 1 
are not repeated here. 



CASS 3 

Situations which are characterized by the structures of Case 1 and 2 
result in the selection and implementation of an activity-design. Soon the 
decision-maker is faced with an evaluative problem — given that the activity 
has been conducted, how well did it do? Case 3 is the a^ posteriori evaluation 
of Cases 1 and 2. 

At first it might appear that the problem should be structured as a test 
of a statistical hypothesis. This is not required, however, since possible 
performance outcomes can be partitioned without statistical consideration. 




f (Kk ) , as per- 



Figure 6 shows the probability distribution of performance, 
ceived, £i priori , by the engineer. 




When actual performance, , falls below the level of K^i.e.^O - Ko 

the engineer has not necessarily made a miscalculation. This performance 
outcome could be attributable to factors which he could not possibly predict 
a priori . The decision-maker may re-evaluate the engineer’s ability to design 
and estimate ; the engineer may evaluate his methods. 

It should be noted, in a precautionary sense, that Ko is a certainty- 
equivalent and the probability of k*. Ck c is dependent in part on the density 
of I C? . The density of K* exists in the same sense that the sampling 
distribution of the sample mean exists even though only a single outcome, , 

is generally observed. It should be added, of course, that much less is 
known about the density of JC .♦ than the density of X • 



If actual performance, , falls between Ko an d > as bhe engineer 
predicted prior to implementation, then it is within the range deemed accept- 
able. The engineer should, nevertheless, evaluate his methods to be sure 
that he has minimized the probability of getting the right answer for the 
wrong reason. 

If actual performance exceeds the upper boundary of the prior distri- 
but ion then the engineer may have miscalculated on the 

favorable side. This prior "pessimism" on the part of the engineer may have 
ruled out other activity-designs which should have been selected for implemen- 
tation. The decision-maker should re-evaluate the engineer’s ability to 
estimate and the engineer should evaluate his methods. 

The decision-maker should understand that the main benefit derived 
from evaluating actual outcomes is discovering improved methods for 

1. designing future activities, 

2. estimating the potential of future designs, and 

3. imp lement ing future activities as they are designed. 

Implementing a newly designed activity often requires a "change 

technology" that is not understood. An activity-design may be perfect with 
respect to technical design, yet its implementation may be difficult due to 
inadequacies in training, support systems, and attitudes. Good communication 
between engineer and implementation team during the activity-design phase 
as well as the implementation phase will improve chances of obtaining high 
performances. 
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Case 3 data types by source* 



Source 



Data Type 



engineer in consultation with 
implementor 



distribution of 



m) 



decision-maker 



evaluator in consultation with 
implementor 



evaluation decision-rules corresponding 
to partition of *0c/) 



performance outcome | 




Only new data types are included. Those which were defined in 
previous cases are not repeated here. 

CASE 4 



Cases 4 and 5 closely parallel Cases 1 and 2. A difference exists in 
that Cases 4 and 5 require the decision-maker to select a package of tasks in 
order to achieve his single objective. 

Case 4, then, considers a decision-maker pursuing a single objective 
and having unlimited resources. The framework is a priori and he wants to 
select a program package consisting of a group of tasks. 

Previously, there was an implied assumption that, in the interval 
to , changes in worth to the decision-maker were proportional to 

changes in performance. That is, 










A Ajft 



Be 



_ a fcxy 



j where, 



s the change in performance from to Kd , 
where both points are between and , 

z the change in worth associated with A fC . 



and 
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It may also be assumed that below K ,the minimum level of acceptable per- 
formance* the same performance-worth relationship exists but the decision- 
maker feels that an inadequate level of worth would result for performance 



is carried through Case 6. 

The reason for introducing the concept of worth, » i- s that the 

decision-maker may feel that various tasks within a program package do not 
contribute equally to the overall objective. Task worth will be used to 
weight task performance, and expected weighted performance will serve as the 
decision-variable . 

The situation confronting the Case 4 decision-maker is that he must 
select a package of tasks. Some task outputs may be independent of outputs 
of previous tasks. Other tasks may rely heavily on outputs of previous tasks. 
The decision-maker may ask his engineers to rearrange tasks in order to 
design an improved package. It was mentioned earlier that the decision- 
maker may perceive the various tasks as contributing unequally to the over- 
all objective. The problem, therefore, is to select the program-package 
promising the largest expected worth. This problem may be exceptionally 
difficult since it not only involves a choice of tasks but also arrangement 
of tasks within the package. 

Theoretically, the engineer would structure a multivariate joint density 
function describing the outcomes and associated probabilites for each program 
package. 



in this interval. The same reasoning holds for K ^ ^ \ j . This assumption 
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For the distribution above, would refer to the minimum acceptable 



performance for task T of activity-design A . , and would refer to the 



.... 9 

’’maximum performance possible" for task T of program package A. . 



Then the engineer would be able to derive marginal density functions 
by integrating the joint density with respect to the other variables over 



their full ranges (e.g.K^.^ to for the general term} . For instance, 



the marginal density of task X is 

\ z V. V 



f, (hr 















Also tne joint distribution of tasks i and % is 



q,>o* / /••■y kku 



•fer. 









1 



If the outcomes of task X are independent of task X outcomes, then their 
joint density function would be equal to the product of their marginal den- 
sities. ^ From the engineer would obtain the conditional 



density of task 2. outcomes given those of task X by regarding as a constant 
over its range of values (for that matter, over any part of its range). 



2 

In a general sense, independence can be assumed when the joint 
aensity can be factored into marginal densities > each involving only one 
variable and with the limits of each not involving another of the variables. 
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If the range of | is restricted then the conditional density of will 
be limited by the range of /(^j . The conditional density of is defined 



as 



T| ( i ) 



As was indicated earlier, the engineer could, if the decision-maker 
requested, answer such questions as "what is the probability tnat performance 
in task Z will take on a value or more given that performance in task! 

was also Ry.or more?" This would be structured as 

i» i . 3 






/ <i fix 

j (fai > i 

fa *%> 




<i 

f, ( k,u)Jfa 






foil 

And if the sequential outcomes are independent, then 

s it. (fa* X kjx ^ 



The preceding type of analysis would be possible from the multivariate 
density level down to the univariate level if the engineer were capable of 
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providing a distribution of such complexity. In all fairness to the engineer, 
however, it is assumed for the remainder of this analysis that he cannot 
develop distributions more complex than bivariate densities. 

Since the decision-maker wants to be able to compare alternative 
program packages so as to select that package promising the highest expected 
worth, he must consider several things. He has the engineer derive the mar- 
ginal probability density for each task from the bivariate density, linking 

each task with the preceding task. The implicit assumption is that dependency 
is an appropriate consideration for adjacent tasks only. This means, for 
instance, if task 2 outcomes depend on those of task £ the dependency would 
be ignored by this analysis. A more general case would involve considering 

^.4 

all task combinations. In this instance, C 3. relationships would result, 3 
but as will be seen later, expected worth is a function of the marginal 
distribution alone. Consequently, the extension is only appropriate when 
considering particular probability questions. Then he assigns a worth value 

t° each task, where • Lastly, he must consider the 

ft* 

relationship of to performance outcomes, , of the respective tasks. 

The procedure involves taking the respective bivariate densities and 
integrating out the marginal probability density, f r to?) , for each task* 
Then^tkcT ) is transformed into , and the expected worth for the 

X * 

entife package can be found by aggregating over the respective tasks in the 
package. For the sake of illustration, let a three task situation be described 
for program package ji. • 5 Ki t ) and 

3 

Appendix A discusses combinatorial aspects of the selection problem. 



When these bivariate densities are factored into their component 



probability densities 



The transformation of |Q j* into j. 


is a linear 


mapping in all instances 


where 


and J(f are psychological parameters. 
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Solving 


for in terms of 


, we obtain 
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The differentials of JCvy are 
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Since the outcomes of task 2. are assumed in our current example, to be 
independent of task 1 outcomes, the expected worth integration involves ob- 
taining the marginal densities, transforming performance into worth, and 
generating the first moment. The marginal density of fC<;| is , and 

it is a probability distribution. That is 



/ --i , o 



The marginal density of * is W, Ur*, ) , and it is a probability 

't I I * » * 

distribution The limits of integration for » are not, in general, O and 
» since the scale of ^ ranges from O to | with and X*.» generally, 

well within this interval. That is 






<1 



1 kt K i 

The expected worth, , of taskj, is 

4 K + ^« 






5 * e J * 

The identical procedure for task 2 will yield » but task 3 » which is 
related to task 2 in terms of performance outcomes, requires a consideration 
of the conditional relationship. The expected value of ff.. given a particular 
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value of is an arithmetic mean. If, however, the expectation of KV 3 

over the entire range of Kii. is desired, then this is a function of /C^ j_ 
and what is needed is the expectation of the distribution of expectations. 
That is: 



1) 6 (YU) is in general a function of X • Let K Cx) denote this 
function and XGObe the probability density of K • Also ){ and Y 
denote the random variables and X and y values of the random 
variables. 



2 ) 
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- £(Y) s 7 

4 

The proof is taken from Mood and Graybill and it shows that the conditional 
expectation of Y |x is equal, in general, to the mathematical expectation of 
the margiral density of Y • Consequently, is equal to the expected value 

of % From this example it is seen that, in general, the expected 



4 

Alexander M. Mood and Franklin A. Graybill, Introduction to the Theory 
of Statistics , 2nd edn. (New York: McGraw-Hill Book Company, Inc., 1963), 

p. 118. 
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worth of program package X, is 
in the package. That is, 

m 

jEI £ Mr ( Mar 

f* 1 l - 



the sum of the expected worths for the tasks 




/ 

j’i ' / . 




As in Case 1, the decision-maker compares alternative packages and selects 
the one with the preferred value of the decision variable."* In this case, 
the decision variable is overall expected worth since he is unconstrained 
by resources and does not have alternative outlets for expenditures. He 
compares overall expected worth for the respective alternatives and selects 
the package with highest expectation. 

In conclusion. Case 4 has introduced several features into the analysis. 
The certainty” equivalence structure was abandoned due to advantages gained 
by using probability distributions for multivariate situations. In addition, 
mathematical expectation played a critical role because it was necessary to re- 
flect the unequal potential contribution •" of tasks. The decision variable, 
weighted expected performance, was obtained by means of transformation. While 
the transformation was treated as linear, the analysis is not restricted to 



Implied in the summation of is an assumption that the scale 

of is an interval scale. This means the scale is unique up to a positive 
linear transformation. That is, a new scale, Y* , can be made by a linear 
transformation of the X scale without distorting the underlying 

relationships. See, for instance, Torgerson, Warren, Theory and Methods 
of Scaling » (New York: John Wiley & Sons, Inc., 1958), pp. 19-20. 
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linear transformations. The perspective needed to transform performance into 
worth, in a given situation, can only be supplied through an understanding of 
the real importance of performance outcomes in relation to the desired outputs 
of the enterprise. 



Case 4 data 

Source 

decision-maker in consultation 
with implementor and engineer 

engineer 

decision-maker and engineer 
engineer providing calculations 



types by source* 

Data Type 

performance to worth transformation 

joint and marginal probability task 
performance densities 

minimum acceptable task performance 

♦ 

expected worth of program package JL 



*Only new data types are included. Those which we: defined in 

previous cases are not repeated here. 

CASE 5 

This case introduces a cost restriction into the structure of Case 4. 

The decision-maker attempts, as in Case 4, to achieve a single objective 
by selecting one program package in an a priori framework. 

Again, as in Case 2, the decision-maker does not encounter an economic 
problem even though he has a cost constraint. The reason for this conclusion 
is that he has no alternative uses for unexpended resources, since he has 
only one objective. The only restriction which the constraint places upon 
the decision-maker is that the package he selects must have an expected cost 



which is less than, or equal to, his budget amount. Hence the program-package 
selected may have a lower expected worth than the program-package selected 
in Case 4. This argument is consistent with the results of Cases 1 and 2. 

Figure 7 is structured similar to Figure 5 (Case 2). A difference 
between the two figures is that Figure 7 f s ordinate is expected worth, while 
Figure 5‘s ordinate is certainty -equivalent. The reasoning is perfectly 

analogous. Admissible points lie in the region where Cj £ ^*A 14 X and 

4 m 

Xi - W/ if \ 

jsi vf\#j , Before continuing, it is appropriate to define 

• I® Case 4 it was seen that each task had a minimum acceptable 
performance level which was set by the decision-maker and the engineer. This 
value, , was actually a concept carried forward from the certainty- 



Total 

Expected 

Performance 





Figure 7. Selecting a Preferred Package with a Cost Constraint 
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equivalent analysis of earlier cases. At a higher order level there is 

still an overall Ka which the decision— maker sets. It was assumed that 

V ^ 

*** was the transformed value of • Therefore % f 

kr 

is assumed to be equivalent to the worth corresponding to the overall 
K . This implies that is a constant for all packages, 

^ 4J 



Ts| 



although the condition is not necessary for the analysis. 

An additional complication arises in that the decision-maker considers 
the advisability of allocating the budget among tasks of the preferred pro- 
gram-package. The position taken by the author is that allocation of the 
budget among tasks belongs in the domain of the implementor since he, alone, 

has responsibility for implementation. The implementor's major problem 
is one of accounting and control. It is true that he also has a problem 
of allocating his budget among tasks. Presumably, he has the available cost 
information prepared by the engineer. It is suggested, in this light, that 
the engineer should involve the implementor in estimating task cost figures. 

A special instance of Case 5 is found in situations where the decision- 
maker pursues a single objective and the various tasks result in production 
of a homogeneous output. For example, consider a planning problem in which 
the decision-maker wants "to maximize the number of high school graduates” 
subject to a budgetary constraint. He will consider alternative insti' 
tutions which produce high school graduates in the same way we have treated 
tasks. His problem may lend itself to linear programming solution. 



James F. McNamara, Pennsylvania Department of Education, is credited 
with pointing out the applicability of linear programming to the special 
instance of Case 5. 
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In summary. Case 5 parallels Case 2. The cost constraint is seen by 
the decision-maker as a restriction on the admissible program-packages. 

Expected program-package worth is the decision-variable taking into account 
the differential ability of tasks to contribute to their end product. 

The only new data type is expected task cost, . It is supplied by 

the engineer, hopefully, in consultation with the implementor. 

CASE 6 

Situations which are characterized by the structures of Cases 4 and 5 
result in the selection and implementation of a program-package. At the end 
of a period of time the decision-maker is faced with an evaluation problem — 
given that a package has been implemented the results should be evaluated. 

Case 6, therefore, is the a posteriori evaluation of Cases 4 and 5. 

The rationale employed in this evaluation is similar to that of Case 
3. The purpose of evaluation is to improve the decision-making process 
through insights into ways to improve the engineering and implementation of 
program-packages . 

The present case has two levels of evaluation. At a higher order or 
overall level the decision-maker wants to evaluate the overall performance 
and compare it to . It is allowable to think in terms of either performance 
or worth depending on the type of question being considered. If an overall 
comparison is required, it is probably more satisfying to transform produced 
worth, based on the observed performance outcomes of the respective tasks, 
into an overall performance variable, . Two values satisfying this 
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relationship are available from the analysis . They are 



overall performance 



produced worth 



\v- f v-.. 



4 iF 



The points connecting 



i- 



s f>>» ' 

and are required of the decision- 






<T < 



maker in order for him to be able to make the overall evaluation. 



For the most part this evaluation is a comment on the manager of the imple- 
mentation team. The engineer also is interested in the outcomes, since it 
was his design that was implemented. The interrelationship of responsibilities 
can cause problems for all parties involved. Therefore, the emphasis should 
be on improvement of methods rather than assignment of blame. 

The second level of evaluation focuses on performance of the respective 
tasks. Here the emphasis is directed more to the engineer’s estimating 
methods, the decision-maker’s worth assignments, and the members of the 
implementation team. These evaluations are similar to those of Case 3 and are 
not repeated here. 

A digression of possible interest is in order. Each task has its own 
probability density. Since only one outcome is observed, the distribution 
of outcomes for taskT is 'fcf £ ) . This, however, is from an <a priori 



point of view. The fact that some "impossible" outcomes can occur, i.e. 



is a clear indication that the notion of a 



° r L • 

posterior distribution of k* is appropriate. Bayesian extensions to 
analysis of the nine cases are not developed, but further work within this 
framework could include such analysis. 



CASE 7 

The structure of Case 7 introduces multiple objectives which will 
be referred to as overall system objectives. The decision-maker is 
faced with the a priori selection of program-packages , herein to be re- 
ferred to as activities, with unconstrained resources. 

For the first time, there are alternative competing objectives but 
since there is no restriction on level of resources, the decision- 
maker will select a system of activities which, when implemented, should 
contribute as much as possible to the overall system objectives. As 
was implied in Cases 1 and 4, the limitations on ultimate system per- 
formance are a function of the engineer's imagination, the state of the 
technological arts, and the abilities of the implementation team. 

Let Oq ) 0# j * • • j Ox j • Oft denote the overall system 
objectives. Since activities selected will ultimately be evaluated on 
the basis of how well they contribute to these objectives, it is necessary 
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to assign values, indicating relative importance, to them. 

These measures of relative importance afford the decision-maker the 
opportunity to consider trade-offs among overall system objectives. The 
common frame of reference or dimension is, of course, value. Assumptions 
underlying this assignment of values are identical to those for assignment 
of worth in Case 4. This value function supplants the wor*h function which 
was used to weight outcomes, so that a common frame of reference may be 
developed. The value function, developed in Case 7, provides a uniform basis 
for assessing outputs prior to, as well as subsequent to, implementation of 
activities. These values are denoted ) ) v (o$ ))•••, V' (Oi yfCi • 

Each activity has potential for contributing to one or more of the over- 
all system objectives. This indicates that the value of overall objective /C , 
V , is the maximum that can be produced by the system with respect 



Russell Ackoff, et al. , show that for systems with two mutually ex- 
clusive, totally exhaustive objectives, the maximization of expected value 
is accomplished by maximizing performance. (Actually they label performance 

as efficiency and define expected value as , where P 4 * is performance 

7 



on objective T by methods, andV-pis the value of objective T.) An extensive 
proof is given for the case with three or more objectives. This maximization 
requires performance and values. 



Russell L. Ackoff, Shiv K. Gupta, and J. Sayer Minas, Scientific Method , 
(New York: John Wiley & Sons, Inc., 1962), pp. 93-97. 



In addition, the above book presents a method for allowing a single 
decision-maker to assign weights to objectives. Other methods for eliciting 
subjective assignments of weights may be found in: 

1. Von Neumann, John and Morgens tern, Oskar, Theory of Games and 

Economic Behavior , 2nd edn. , (Princeton, N.J.: Princeton University 

Press, 1947). 

2. Helmer, Olaf, Convergence of Expert Consensus Through Feedback , 

(Santa Monica: The Rand Corporation, September 1964). 
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to • If all overall objectives were produced perfectly, the system 

would have produced the maximum value possible, i e. , v 

VtoA 







The extent to which a particular activity contributes to a particular 
overall system objective is a function of two variables — potential for 
contribution and actual activity performance. That is, C 5 ti [fa ,vM 

e* 

where the effectiveness of the /Jk TW activity toward the 

overall system objective, 

the maximum value that the^^** activity could produce 



and 



*T#» 



fa- = 



with respect to the overall system objective, 



**4 - the adjusted performance score, 



tn 

The maximum value that the activity could produce with respect to the 



overall system objective is a function of how important that activity is 
relative to the other activities aiming at . Here the decision-maker 
makes a relative value assignment such that the sum of the relative weights 
is equal to v(o-) 

. (Were there only one overall objective, there would 
be no need for and Case 7 could be reduced to Case 4.) The distribution 

of overall value to activities is represented as follows: 
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Based on this distribution of potential value, Wo; ) , to activities, it 



is now possible to determine effectiveness at the activity level. To do this 
several performance concepts need definition. 

The decision-maker and the engineer establish a criterion that can be used 
to assess the package’s performance. Then they list the possible outcomes and 
relate these possibilities to index scores for the criterion. For instance, 
suppose that an activity was directed to "getting eligible people to pass 
Examination Five." One criterion might be the number of eligible people pass- 
ing the examination expressed as a percentage of the total number of eligible 
people. The index could be formed by the percentage points. In this context, 
is the index score (percentage) reflecting the performance of package 
toward overall system objective ju . The decision-maker, however, may not feel 
that each percentage point is entitled to one percent of the potential value 

jj, >\ 

of aW* this case^lC^^w is transformed into KU- The relationship between 



Figure 8. Transformation of Performance Index Scores into Effectiveness Scores 



K*. • , K* • and is depicted in Figure 8. When • is multiplied 

£ 

by the result is or effectiveness. Effectiveness is defined as 
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Performance 

LOt 
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Value Produced or , 
Effectiveness Score! 
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The right hand section of Figure 8 shows the relationship between the 



underlie the transformation. 

The left hand section of Figure 8 shows the way 1C * is used to 
generate value produced or effectiveness. The reflection line joins the 
points , **<• »oJ and s »l } , and specifies 

that the change in effectiveness with respect to a unit change in performance 
is constant. The relationship is 



Now that effectiveness has been defined as weighted performance, two 
simple ideas should be considered. 

1. Are large effectiveness scores always preferred? 

2. What is the maximum effectiveness score? 

In order to conclude that the decision-maker prefers any larger £ score 
to any smaller £ score, several comments are necessary. First, the relative 
values assigned to the overall system objectives represent a set of targets for 



criterion index scores, , and the decision-maker 1 s adjustments, lc* • . 

. jL S\ 

A special case occurs when lc.sk* .over the complete range from o to |.o . 




f » where the function.^, denotes the perceptions that 




and the change in effectiveness for a small change in performance is 






with the differential of effectiveness. 



) equal to 



^ t m p>m< ■< !,***!« W 
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the decision-maker,. If a unit of value is equally preferred, no matter what 
objective is being produced, the decision-maker clearly prefers a larger 



to a unit of value of another objective then the assignment of values requires 
revision. It is, of course, possible to constrain the production of an overall 
objective in such a way that some production is preferred over other production. 



could have efficiency implications for the system. It would have the same 
effect on the system operations as a policy, since a policy can inhibit func- 
tioning with respect to given criteria in selected instances. 

A production function shows technical possibilities relating alternative 
inputs to the output or commodity under consideration. For instance, test 
score achievement of a school district may be viewed as the commodity under 
consideration. Conceptually, this could appear as 



Y is a vector of school variables, 

2 is a vector of teacher variables. 

The search for an optimal solution for the Case 7 type of problem depends 
on such factors as the nature of the production function, the costs of produc- 



to a smaller £ in any case. If a unit of value of one objective is preferred 



For example, the decision-maker could specify that overall objective A, must 



be produced at some minimum level or better, i.e. 




. This type of production constraint 




where T is test score mean, 

X is a vector of non-school variables 



tion, performance. 



and values 
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The maximum £ score is the sum of the valuations assigned to the overall 
system objectives, i.e., . This is readily seen by inspecting 



c* 

thefc function, 

£*.=£*. vifo. •) £?» 

'J'K* -wh * # , and observing that when K" • is 

equal to 1,(7 (the maximum value since 0 ^ ^ I*®), a ^(<1^ ) • 

Therefore, under conditions of perfect performance, s l-°> 

V(0/), and over all objectives the maximum £f *=£ W<>v) . 

* 

In this situation, as in Case 4, it is within the scope of the argument 
to treat performance as a random variable. This procedure is now outlined. 

The engineer provides the probability density of as in earlier 

instances. In an a priori sense, the distribution Li(kL) is identical 

w * Two transformations are required. The first allows scores 

»a 

to substitute for • scores. Since a simple multiplicative form has been usee 

to relate B to its components , the second transformation is constant for all 

C* 

such relationships and the probability density of C can be inferred from 

That is « 

£*• 3 £ • [ $*■ , v («*<)] 

'*»< *- 'hit ) «J 

E*.* &WO 






'VW 






_ 1U «■» 

Since is a constant for all > the expected value, £^4, 






is given by 
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Case 7 introduced many of the key concepts which serve as a foundation 
for the more realistic Cases 8 and 9. The co capt of overall system objective 
was introduced into Case 7. With this notion we are able to recognize that 
conflict may appear within an organization as a function of the multiple and 
competing objectives. In order to explicitly acknowledge differences among 
the overall objectives, the decision-maker assigned weights reflecting their 
relative importance. 

A set of coefficients was derived in order to determine the relative 
ability of an activity to contribute to overall objectives. 

Several performance concepts were defined in order to translate performance 
criterion outcomes into a performance index, expressing the degree to which 
task performance was obtained. 

Lastly, an effectiveness variable was defined. It combined performance 
index score with potential to contribute, yielding value produced. 

As the situations become more realistic it becomes increasingly difficult 
to specify who supplies various data types. Consequently, let it suffice 
to say that useful data are usually not derived "in a vacuum" e.g. a research 
off i' e which tells a program director: "you need this information, so here 

it is." T? tiless those parties who "need" the data believe they need it and 
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want to use it, data will be derived "in a vacuum «" 

CASE 8 

Case 8 represents an extension of Case 7 under a budget constraint. Here 
the decision-maker pursues a set of overall school system objectives by means 
of selecting a group of activities in an ja priori framework. This is the first 
and only a priori cost-effectiveness case for school systems. Analysis of 
single objective situations is sub-optimal. Unfortunately, sub— optimal 
decision models are expedient and familiar. Comprehensive or system-wide models 
are virtually nonexistent in local school districts. 

Cost-effectiveness is considered by the author to be the general way to 
approach school district planning and decision-making. Let us see why. 

Benefit-Cost Analysis is applicable to the selection of projects from 

a set of alternatives when investment is clearly the spirit of the decision, 

and when inputs and outputs can be fairly well measured by dollar units. The 

method derives from the criterion of present value of net benefits as a basis 

9 

for comparing proposed investment alternatives . 

If the decision— structure is suitable to the design of a totall v new 
system with a remote time horizon, then ’’benefit-cost analysis" may be 
appropriate. On the other hand, when the decision-structure is suitable for 
planned and incremental improvement of an existing system with a time dimension 
weighted heavily toward the present and immediate future, then cost— ef Eectiveness 
methods are appropriate. Cost-effectiveness, as a school district planning 
tool, i 3 further justified by its superior ability in handling intangible and 

9 Clear statements of alternative benefit-cost criteria may be found in: 

McKean j Roland N. Efficiency in Government through Systems Analysis 
(New York: John Wiley & Sons, Inc., 1958). 

1 

me 



incommensurable outputs which derive from the educational processes. 

Previous developments have provided clues as to the necessary and sufficient 
conditions for economic analysis. Case 1 was not an economic situation because 
costs were not involved. In Case 2 it was observed that even the presence of 
a cost constraint was not sufficient to suggest economic analysis since the 
decision-maker was interested in a single objective and he therefore had no 
alternative outlets for funds that might be saved due to economic considerations. 
Case 5 could, in the special instance for which linear programming was applicable 
use economic analysis, but this was largely a comment on level of application. 
That is, the engineer could use linear programming to sort alternative packages 
ror the decision-maker, but at the higher order level of the decision-maker 
there is one objective and a constraint. The higher order problem is solved 
without recourse to the linear programming model or any other form of economic 
analysis. 

Consequently, it is not until Case 8 that the decision-maker finds budgetary 
restrictions and multiple outlets (objectives) competing for this budget. The 
necessary and sufficient conditions for cost-effectiveness analysis to be 
appropriate are: 

1. More than one alternative outlet for the available resources. 

2. Less available resources than are required to satisfy all of the 
alternatives. A budget constraint is not sufficient unless it 
forces the curtailment of at least one activity, e.g., a budget 
constraint of $1,000,000 would probably not restrict vacation 
plans of a married couple. 

Case 8 borrows the structure and developments of Case 7 and adds to these 
the analysis of cost. The critical relationships are among performance, cost, 
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and non-cost factors. That is. 




= k ( c , G • ) 

' ) /)rv< / 



on\/l 



where : 



the performance index score 




• the estimated cost for activity as a producer 

(this estimate is not treated as a random variable), 

** the non— cost factors associated with the performance 
of activity as a producer of . While these could be 
viewed as a function of , they are assumed to be given 

and adequate in the analysis. They include the human 
factors, such as disposition toward the tasks, level of 
managerial know-how, level of skills, and the technological 
factors. 



. To interject some realism into the analysis, assume that the production 



maker has his engineers develop cost-effectiveness curves for each proposed 
activity. Figure 9 shows a hypothetical curve prepared by the engineer. 
This function indicates that dollars must be spent before any production 




of 0 ^ is specified by the decision-maker to be at least . 



The decision- 






/ 




Effectiv 




Figure 9. An A Priori Cost-Effectiveness Curve, 



-44- 



is realized. When dollars are spent the marginal productivity response 



factors are restricting further productivity. Implicit in the construction 
is a series of relationships. They are the relationships between 



If the system had been in operation for a period, then evaluation could 
be conducted for the purpose of obtaining cost-effectiveness points. These 
points would reflect the productivity of the system as it is presently defined. 
Allocations for the next period could then be based on these considerations. 

The actual solution to the problem involves non-linear programming with 
three sets of inputs: 1) the functions relating effectiveness to cost, e.g«* 



problem would involve a numerical analysis of the functions. This could be done 
by taking the derivative of effectiveness with respect to cost at, for example, 
$100 inte vals for each proposed activity. A list of slopes would be generated 



/ r # 

becomes almost zero at the level cr • This implies that the 









f* . — X* . f C * \ .2) the specifications (policy constraints) 

~ vmvt ' / 

telling how much of which objectives should be produced, e.g.^C*^ £ 
and 3) the budget constraint, i.e., 2 . ±e . The programming 



which could be combined for all activities. Production inequalities CuK> se 
specifying production must be at least ^ ) could be satisfied first. Once 






this was done, the remainder of the problem would consist of a search for the 
maximum productivity for the unencumbered funds. This is not as simple as the 
procedure just described implies, because the decision-maker cannot purchase 
the increment of effectiveness related to the expenditure of $100, in the 
interval $2,400 to $2,500, unless he has also agreed to purchase the first 
$2,400 of the activity 1 s production. 

Case 8 represents a situation which is faced by school district chief 
administrators. The prescription emerging from detailed analysis of this 
recurrent situation is rational planning. ^ Case 8 introduces the concept of 
efficiency into the model. The efficiency variable is defined as effectiveness 
per unit of cost. 

The principal data files of Case 8 can be described as follows: three 

are structural (overall objectives, activities, and coefficients which link 
activities to overall objectives); two are judgmental (performance criteria 
and value assignments); and two are outcome files (performance and costs). 

It may be noted parenthetically that experience in this kind of planning 
with schools has suggested three other files: pupil needs assessment, socio- 

political survey information, and an alternatives file used by "engineers" 
within the school district. 

Case 9 allows for evaluation of the school district after the school year. 
Realistically, Case 9 feeds information to Case 8 in an iterative process. 



method has been developed called "Comprehensive Planning" and is 
based on deterministic operationalization of Cases 8 and 9. 
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CASE 9 



The final case considers the evaluation of Cases 7 and 8. It is the gen- 
eral problem faced by the decision-maker who has an ongoing system of activi- 
ties and wants to evaluate current operations so as to prescribe subsequent 
action. 

The evaluation of Case 7 (no constraint on resources) is similar to Cases 
5 and 3 and is not repeated here. The one difference is that the system of 
Case 7 has multiple objectives and therefore one would encounter more o. -he 
same comparisons. 

Analysis of Case 9 leans heavily on the concepts and constructs of Cases 
7 and 8. Overall system objectives and values provide the production target. 
What information is required in order to determine how well the system pro- 
duced? In an overall sense it is the sum of the production (effectiveness) 
scores for the various activities . This total compared to the maximum score 
possible rates the system in terms of relative effectiveness. That is. 




s / in the 

denominator 




* ** 

R 



The relative productivity of the system toward overall system 

Q 1— jt ^ M Jf 
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Overall evaluations do not provide sufficient information to evaluate for 
the purpose of making future-period decisions. Presumably, decisions made in 
the present for the next period should incorporate the same kind of analysis 
as was used in Case 8; that is, a systematic evaluation of alternative possi- 
bilities so as to arrive at a combination of activity expenditure levels which 
will lead to a higher level of productivity. 

This calls for an assessment of the cost-effectiveness relationship for 
each ongoing activity. An assumption is made that activities will be continued 

although it is not necessary for the analysis (the reason for this assumption 
is to avoid the consideration of newly developed alternative activities). A 
cost-effectiveness curve is obtained for each activity by utilizing the curve 
developed to plan the activity in Case 7. It is indeed possible that the im- 
plementation staff may have contributions and modifications to make for these 
prior curves. 

Figure 10 shows a cost-effectiveness curve for activity The curve 

has been revised from the a priori relationship presupposed by the analysis of 
Case 8. Several aspects of the relationship warrant attention. With regard 
to costs, ‘ is the amount of resources actually expended and the level 




Figure 10. 



Revised Cost-Effectiveness Curve* 
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allocated to the activity to pursue overall objective X • Practically, 

C <• C* 

v /m* /•»< probably be close to zero since administrators are re- 

luctant to return unexpended resources. Organizations who are able to 
make administrators comfortable with returning resources that cannot be 

spent wisely will benefit substantially. This problem is probably more 
acute in public sector organizations. If the activity were a develop— 
mental activity , then the expenditure of Ox — C • additional dollars 
would allow productivity to rise to £Fy (assuming the adequacy of the 
revised fV. s If the activity were, on the other hand, con- 

cerned with the maintainance of an ongoing activity , then it would 
require dollars to produce at a level of fj, . Since the purpose 
of this analysis is to evaluate ongoing educational systems , the develop- 
mental activity problem is disregarded. It must be underscored, however, 
that incremental planned change, based on incremental movement from 
presently defined activities , is probably not going to meet all school 
district needs. Alternatives to the present system must also be 
included in the planning process. 

An inspection of the cost-effectiveness curve such as the one in 



ductivity during the current year. The implementation staff should have a 
reasonably good idea of how production could be changed were small changes 
in level of activity suggested. They may be less certain concerning large 
changes from the present expenditure level. The engineer, on the other hand, 
has studied the production function and, consequently, is more aware of the 




ERiC 



process that relates the inputs and outputs than the implementation staff. 
Consequently, the engineer should be available for consultation (if not 
direct involvement) in the process of assessing changes in productivity were 
costs to be varied. The difficulty is that the engineer is not responsible for 
subsequent implementation and maintenance of the activity. This suggests 
that those responsible for these evaluations should consult the engineer. 

At any rate, the implementation staff should prepare a schedule that 
lists expenditure levels above and below C *as well as the estimated 
responses in production associated with the changes from G^.: . These 

Je ■ 

are analogous to the ** values generated in Case 8. 

Ic~ 

The same type of evaluative process is instituted as in Case 8. There 
is, however, the problem of what will be referred to as human factors, H • 

When resources are to be taken away from one activity and given to another 
based on cost-effectiveness considerations, then the decision-maker should 
be painfully aware of such human elements as loss of morale. While no attempt 
is made to study or measure j-f , it does figure into the decision process. 

In general, if 
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then resources will be 
inequality the symbols 



diverted from activity CL .to activity $ • . 

hi ***< 



are defined as 



In this 




the change in performance in year £-f I 
associated with the cost increment if it 
would be added to CL • 

the decision-makers assignment of potential 
value to activity as it would contribute 
to overall objective^' . 




the increment of resources that is being 
considered for transfer from 



•W* a dollar unit assessment of the human factors 
associated with the loss of AC ^4 
dollars by the administrator of activity 
CK ¥\\ and the gain of these dollars 
by the administrator of activity 
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METHODOLOGICAL SUMMARY 

Cases 8 and 9 have drawn from the other cases in order to provide 
a general planning, evaluation, decision framework which is applicable 
to ongoing school systems. Certainly, there has been no implication 
that the models developed in this monograph are understandable to or 
useable by school administrators. To the contrary, the Case 8-9 
model is far from an operational reality capable of being implemented 
in school districts. 

Before we add a few final comments it is appropriate to summarize 
the 9 Cases. Case 1 provided a starting point in terms of certainty- 
equi .fence and produced a criterion for selecting an activity directed 
toward one objective with no constraining factors. Case 2 instituted 
a cost-constraint and showed that a possible result could be the elim- 
ination of the preferred activity of Case 1 because it costs too much. 
Case 3 indicated that the evaluation of Cases 1 and 2 was not essentially 
statistical in nature. It stressed that the purposes of evaluation were 
derived from the need for improvement in future engineering design and 
estimation, decision-making and implementation. 

Case 4 considered multiple activities functioning to meet a single 
shared objective. The decision-maker’s need for assigning weights to 
outcomes according to their worth was indicated. Case 5 brought costs 
into the Case 4 context. The implication of a cost constraint was that 
some packages, deemed acceptable on performance grounds, should be 
eliminated from consideration by the decision-maker. Cost versus 




effectiveness was still not necessary for analysis since there was only 
a single objective in the Case 5 structure. It was seen, however, that 
linear programming, and other programming models, was an admissible meth- 
odology for lower order problems which the engineer must solve. Once, the 
engineer makes his selections , the decision-maker does not face an economic 
problem. The evaluation of Cases 4 and 5 in Case 6 was similar to Case 
3 with the emphasis again placed on evaluation for decision-making and 
future improvement in operations. 

A set of overall objectives was introduced in Case 7. The decision- 
maker considered how to select a se*- of activities', to produce these objec- 
tives. Productivity was defined as a function of performance and potential 
value. Case 8 provided a cost-effectiveness framework to facilitate de- 
cisions among proposed activities designed to meet the set of overall objec- 
tives. Benefit-cost analysis was seen as being appropriate when investment 
was clearly the spirit of the analysis. It was underscored that the decision- 
maker only has an economic problem when he pursues at least two objectives 
and has a level of resources that restricts at least one activity that could 
be conducted. 

Case 9 tied the cost-productivity relationships which had been evaluated 
for the current year and related them to the budgetary process for the next 
year (really a new Case 8). The development of cost-effectiveness curves 
to evaluate ongoing operations was central to relating Case 9 evaluations 
to Case 8 decisions. Perhaps the most critical and least performed role is 
that of educatic.v.1 engineering - - the function which describes alternative 
processess which relate inputs to outputs in education. 



A few final comments are required. The treatment in this monograph 
has run rampant over many real world technical difficulties. There are 
many problems associated with measurement of key variables including 
educational processess, performance criteria, costs, and value assignments. 
These are confounded by a great deal of uncertainty associated with "change." 
Think, for example, of the problems of convincing a school district business 
manager that it would be beneficial for his superintendent (and him) to 
change their line-item accounting system to a system which tracks costs 
by objective, program and activity. And if the business manager saw that 
this was good, how could he effect the required changes and still maintain 
legal and financial control. 

The Administering for Change Program of Research for Better Schools 
has been working on making this model operational for the better part of 
four years. The method which has evolved is called "Comprehensive Planning # " 
It is an attempt to put the Cases 8-9 iterative process into a form ~.*hich 
may be used by managers of school districts to facilitate their plannii g 
efforts. The most difficult problems arise out of change itself and getting 
people to deal with change. Until change technologies are more clearly 
specified, there are serious doubts about the feasibility of prescriptions 
such as Comprehensive Planning. ^ 

^Temkin Sanford, Comprehensive Planning for School Districts , 
American Education Research Association Convention, "Planning and Managing 
Changes in Local School Districts Symposium," Minneapolis, March 1970, 
(Mimeographed . ) 
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Appendix: Combinatorial Aspects of Project Selection 



Suppose the educational engineer submits feasible projects 
' l J " V ^ ^ ) to the decision-maker. Each project is accom- 
panied by an estimate of cost, C ' £ , and its expected contribution to each 
overall objective, • Objectives are denoted: f\ # ... , . /? . 



The estimated total contribution of all projects is 

A 






t . * 

The problem is to select the set of projects which maximizes V 

A . /, A 



^ O. ' . 

subject to C — O* The symbol C is total cost and £ is budget. Struc- 
turing the problem as we have oversimplifies it in several respects. First, 
probabilistic considerations for and have been ignored. Second, 
complex projects must be evaluated in terms of tasks and often sub-tasks. 



Third, situations are complicated by constraints, such as time, in addition 



to budgeting restrictions. 



If we can purchase projects in continuous amounts, e.g. f 10.0% of project 
1 and 40.5% of project 2, standard linear programming techniques are applic- 



able. If, as is more common, we have to either purchase or not purchase a 
project in its entirety, then we have a combinatorial problem. Actually, 
the case is usually in between; we can select a project at a few discrete 



levels but not as a continuous variable. This, then, is still a combinatorial 
problem . 



W) 



or credit for this Appendix belongs to Roger L. Sisson. 



-55- 






For example, if there are 20 feasible projects, there are 

20 

2 -1 or over 1,000,000 possible combinations. A combinatorial 

W 

2 

problem can be set up as an integer linear programming problem. 

It is also possible to approximate the optimal set by sampling 
from the set of possible sets. The best set from a sample of a few 
hundred sets can be shown to have a performance which is close, 

3 

in a probabilistic sense., to the best possible set. 



^Weingartner , H. Mathematical Programming and the Analysis of 
Capital Budgeting . (Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1963). 



3 

See: Sisson, et al. "The Project Concept in Planning, Programming 

and Budgeting," Journal of Socio-Economic Planning Sciences (a forthcoming 
issue) . 

Clough, D. J. "An Asymptotic Extreme Value Sampling Theory for Estimating 
Global Maximum," CORS, Vol. 7, No. 2 (July 1969), pp. 105-115. 
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